Information processing apparatus, information processing method, and computer program

ABSTRACT

An aspect of the present disclosure includes acquiring an image representing a shadow component in an image capturing environment, the shadow component being reflected in a multi-valued image obtained by capturing an image of a subject; specifying an area having a luminance greater than a predetermined luminance value, the area being included in the image representing the shadow component acquired in the acquiring; correcting the image in such a manner that a luminance of an outer peripheral area of the specified area deceases; and generating a binary image by performing binarization processing on a pixel value of a pixel of interest in the multi-valued image based on a pixel value at the same coordinates as those of the pixel of interest in the corrected image.

BACKGROUND Field of the Disclosure

The present disclosure relates to an image processing technique for aninformation processing apparatus.

Description of the Related Art

Binarization processing for converting an image into a monochrome binaryimage by determining whether each pixel value of input color or grayscale image data is greater than a threshold is known. Japanese PatentLaid-Open No. H04-290177 discusses binarization processing using asingle threshold for each pixel of image data,

In Japanese Patent Laid-Open No. H04-290177, a shadow component in animage capturing environment that is reflected in image data is not takeninto consideration in binarization processing. Accordingly, when theshadow component included in the image data is darker than thethreshold, the shadow component is converted into black as a result ofbinarization. Therefore, in the case of capturing an image of a documentof a blank sheet including a shadow component, the area of the blanksheet is converted into black. If illumination such as fluorescent lightis reflected in image data, binarization processing on an outerperipheral area of the illumination cannot be accurately performed.

SUMMARY

According to an aspect of the present disclosure, an informationprocessing apparatus includes an acquisition unit configured to acquirean image representing a shadow component in an image capturingenvironment in which a multi-valued image obtained by capturing an imageof a subject is reflected; a specifying unit configured to specify anarea having a luminance greater than a predetermined luminance value,the area being included in an image representing a shadow componentacquired by the acquisition unit; a correction unit configured tocorrect the image in such a manner that a luminance of an outerperipheral area of the area specified by the specifying unit decreases;and a generation unit configured to generate a binary image byperforming binarization processing on a pixel value of a pixel ofinterest in the multi-valued image based on a pixel value at the samecoordinates as those of the pixel of interest in the image corrected bythe correction unit.

Further features of the present disclosure will become apparent from thefollowing description of exemplary embodiments (with reference to theattached drawings).

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a system configuration diagram according to one or more aspectof the present disclosure.

FIGS. 2A and 2B illustrate an example of appearance of a mobileterminal.

FIG. 3 is a block diagram illustrating the mobile terminal.

FIG. 4 is a block diagram illustrating a digital multifunctionperipheral.

FIG. 5 is a flowchart illustrating a first exemplary embodiment.

FIGS. 6A to 6C illustrate an example of image binarization processingaccording to one or more aspect of the present disclosure.

FIG. 7 is a flowchart illustrating threshold map generation processingaccording to one or more aspect of the present disclosure.

FIG. 8 is a flowchart illustrating reflection countermeasure processing.

FIGS. 9A and 9B are flowcharts illustrating a relationship between anoperation and processing according to one or more aspect of the presentdisclosure.

FIGS. 10A to 10D illustrate an operation screen.

FIGS. 11A to 11H illustrate an example of an image under binarizationprocessing.

FIGS. 12A to 12H illustrate a binarization threshold.

DESCRIPTION OF THE EMBODIMENTS

Modes for carrying out the present disclosure will be described belowwith reference to the drawings.

First Exemplary Embodiment

FIG. 1 illustrates an overall configuration of a system suitable for afirst exemplary embodiment of the present disclosure. A local areanetwork (LAN) 100 is connected to each of a wireless router 102, acomputer 104, and a digital multifunction peripheral 300. A mobileterminal 200 is connected to each of the digital multifunctionperipheral 300 and the computer 104 through the wireless router 102 andthe LAN 100. The mobile terminal 200 and the digital multifunctionperipheral 300 functions as an information processing apparatus in thefollowing exemplary embodiments.

FIGS. 2A and 2B illustrate an appearance of the mobile terminal 200 usedin the present exemplary embodiment. While there are various types ofmobile terminals 200, FIG. 2A illustrates an example of the frontsurface of the mobile terminal 200. The front surface of the mobileterminal 200 is provided with a touch panel display 201 and an operationbutton 202. Specifically, the touch panel display 201 is a deviceincluding a combination of a display device for displaying information,buttons, images, and the like necessary for operation and a device forinputting a position by being touched by a finger of a human or thelike. FIG. 2B illustrates an example of the back surface of the mobileterminal 200. On the back surface of the mobile terminal 200, an imagecapturing unit 203 is disposed. In the present exemplary embodiment, themobile terminal 200 can be applied to any terminal device having acamera function including the image capturing unit 203. Specifically,the present disclosure can be applied to a smartphone, a mobile phone, atablet terminal, and a laptop computer having a camera function, adigital camera having a communication function, and the like. Inaddition, the present disclosure can be applied to a document cameracapable of capturing an image of a document in a non-contact manner.

FIG. 3 is a block diagram illustrating an internal configuration of themobile terminal 200. However, this block diagram is an example of aconfiguration for carrying out the present exemplary embodiment.Referring to FIG. 3, a central processing unit (CPU) 210, a randomaccess memory (RAM) 211, and a read only memory (ROM) 212 transmit andreceive programs and data through a data bus 220. The data bus 220 isconnected to each of a storage unit 213, a data transmission/receptionunit 214, an image capturing unit 215, a display unit 216, an operationunit 217, an image processing unit 218, and a motion sensor 219. Thesecomponents together with the CPU 210, the RAM 211, and the ROM 212transmit and receive programs and data.

The storage unit 213 is a flash memory that stores image data andvarious programs. The data transmission/reception unit 214 includes awireless LAN controller to implement transmission and reception of datato and from the digital multifunction peripheral 300 and the computer104 through the wireless router 102. The image capturing unit 215corresponds to the image capturing unit 203 described above and acquiresa captured image by capturing an image of a document. Data on theacquired captured image is processed by the image processing unit 218and the CPU 210 through the storage unit 213 and the RAM 211. Afterthat, the CPU 210 performs processing on the processed image data, suchas display of the processed image data on a display of the display unit216, storage of the image data in the storage unit 213, and transmissionof the image data to an external device through the datatransmission/reception unit 214.

The display unit 216 is a display constituting the touch panel display201 described above. The display unit 216 performs live view displaywhen a camera function is used to capture an image of a document, anddisplays image processing results according to the present exemplaryembodiment, a notification about a processing process, informationnecessary for operation, and the like.

The operation unit 217 includes a touch panel constituting the touchpanel display 201 described above and the operation button 202, receivesan operation from a user, and transmits information about the operation.

The motion sensor 219 includes a three-axis acceleration sensor,electronic compass, and a three-axis angular velocity sensor mountedthereon, and is capable of detecting a posture or movement of the mobileterminal 200 by using known techniques.

The CPU 210 executes programs held in the ROM 212 and the storage unit213 to thereby control components in the mobile terminal 200.

FIG. 4 is a block diagram illustrating an internal configuration of thedigital multifunction peripheral 300. However, this block diagram is anexample of the configuration for carrying out the present exemplaryembodiment. Referring to FIG. 4, a controller unit 310 is connected to ascanner 301 which is an image input device and a printer 302 which is animage output device. The controller unit 310 is also connected to theLAN 100 and a public line 400, and inputs and outputs image data anddevice information to control the entire system. A CPU 311 functions asa controller that controls the overall digital multifunction peripheral300. A storage unit 312 stores image data and compressed data andincludes a system work memory for allowing the CPU 311 to operate. Anetwork I/F 313 is connected to the LAN 100 to input and output data. Amodem 314 is connected to the public line 400 and performsmodulation/demodulation processing for transmitting and receiving data.An operation unit I/F 315 is an interface unit with an operation unit303, and outputs image data to be displayed on a display unit (notillustrated) on the operation unit 303 to the operation unit 303.Further, the operation unit I/F 315 has a function for transmittinginformation input by the user through the operation unit 303 to the CPU311. An image processing unit 316 performs correction, processing, andediting on the image data externally input through the scanner 301, thenetwork I/F, or the like. A compression/decompression processing unit317 compresses and decompresses the image data by a predeterminedcompression method. A device I/F 318 connects the scanner 301, theprinter 302, and the controller unit 310 to each other and performsimage data synchronous/asynchronous conversion. The CPU 311, the storageunit 312, the network I/F 313, the modem 314, the operation unit I/F315, the image processing unit 316, the compression/decompressionprocessing unit 317, and the device I/F 318 are connected to a data bus320.

The scantier 301 irradiates a document with light, uses a lightreceiving element, such as a charge-coupled device (CCD) line sensor, toread the reflected light, and converts the light into an electric signalrepresenting an image on the document, thereby generating digital imagedata. The printer 302 forms an image on a sheet based on image data, andoutputs the sheet. Examples of the image formation method used in thiscase include an electrophotographic method and an inkjet method.However, any format may be used.

The digital multifunction peripheral 300 having the configuration asdescribed above is capable of, for example, printing image datatransmitted from the mobile terminal 200 through the wireless router 102and the LAN 100 by using the printer 302, and transmitting a facsimilethrough the modem 314.

FIG. 5 is a flowchart illustrating image binarization processingaccording to the present exemplary embodiment. When the mobile terminal200 acquires, as input image, an image captured by the camera functionincluding the image capturing unit 203, an image stored in the storageunit 213, an image received from an external device, or the like,processing is started.

In step S501, the image processing unit 218 converts an input image,which is a multi-valued image, into a gray scale image, If the inputimage is a gray scale image, the processing of step 5501 need not beperformed. In step S501, when the input image is a color image withthree channels of red (R), green (G), and blue (B), an image with asingle channel is generated by mixing the three channels at apredetermined ratio. The mixing ratio used in this case is notparticularly limited. For example, a gray scale image is generated by anNTSC weighted average method. The input image used herein refers to animage in which a shadow component in an image capturing environment isreflected in a multi-valued image obtained by capturing a documentimage.

Next, in step S502, the image processing unit 218 acquires a thresholdmap. The threshold map refers to an image representing a shadowcomponent in an image capturing environment that is reflected in aninput image. In the acquisition of the threshold map, a gray scale imageobtained by capturing an image of a blank sheet in an image capturingenvironment, which is, for example, preliminarily stored in the storageunit 213, may be read. Alternatively, the threshold map may be obtainedby analyzing a feature amount relating to a brightness of an input image(multi-valued image) and estimating a shadow component reflected in theinput image. In the present exemplary embodiment, assume that thethreshold map is estimated mainly based on the input image describedabove, and threshold map generation processing will be described indetail below with reference to FIG. 7.

Next, in step S503, the image processing unit 218 generates a binaryimage from the gray scale image and the threshold map, For convenienceof description, assume herein that each of the gray scale image and thethreshold map is composed of pixel values representing luminancecomponents represented by 8-bit pixels, and a pixel value 0 representsblack and a pixel value 255 represents white. Also assume that a binaryimage to be output is composed of pixel values represented by 1-bitpixels, and the pixel value 0 represents white and a pixel value 1represents black. Each pixel value of the gray scale image atcoordinates (x,y) is represented by g(x,y). Each pixel value in thethreshold map is represented by t(x,y). Each pixel value of the binaryimage to be output is represented by b(x,y), As shown in Formula (1),the image processing unit 218 compares the pixel value g(x,y) of thegray scale image with the pixel value t(x,y) of the threshold map at thesame coordinates, thereby determining whether the pixel value b(x,y) ofthe binary image is 0 (white) or 1 (black) to generate a binary image,

when g(x,y)≥t(x,y), b(x,y)=0

when g(x,y)<t(x,y), b(x,y)=1   Formula (1)

Specifically, when a pixel value representing a luminance component of apixel of interest in the gray scale image is equal to or greater than apixel value representing a luminance component at the same coordinatesas those of the pixel of interest in the threshold map representing ashadow component, a pixel value representing white is generated. Whenthe pixel value representing the luminance component of the pixel ofinterest in the gray scale image is smaller than the pixel valuerepresenting the luminance component of the pixel of interest in thethreshold representing the shadow component, a pixel value representingblack is generated. In step S503, binarization processing is performedon all pixels of the gray scale image, thereby generating a binaryimage.

As described above, binarization suitable for each pixel of the inputimage is performed using the threshold map.

FIG. 6A illustrates an input image which is composed of an object (acharacter string of “A” to “M” in this case) to be reproduced in blackduring the binarization processing, and a background to be reproduced inwhite. A lower left portion of an image of a subject is darker than anupper right portion of the image due to the effect of a shadow formedduring image capturing, although the background is uniform. The elementof nominiformity in brightness caused (hiring image capturing ishereinafter referred to as a shadow component. FIG. 6B illustrates thethreshold map corresponding to the input image. FIG. 6C illustrates aresult of performing binarization processing on the input image usingthe threshold map, and also illustrates an example in which a shadowformed during image capturing is not reproduced and only the characterstring that is an object to be reproduced is converted into black.

FIG. 7 is a flowchart illustrating a processing flow in which the imageprocessing unit 218 estimates the threshold map based on the input imagein step S502 illustrated in FIG. 5.

FIGS. 11A to 11H illustrates an image example for describing theprocessing of the flowchart illustrated in FIG. 7. FIG. 11A illustratesan input image. The input image includes a bright portion in whichillumination light, outside light, or the like is reflected during imagecapturing. In FIG. 11A, a brighter area due to reflection is describedas a reflected area, text or the like to be reproduced in black on asubject is described as an object, and a background with nonuniformityto be reproduced in white is described as a shadow component.

In step S701, the image processing unit 218 segments the input imageinto a plurality of blocks. In this case, the shape of one block and thenumber of input pixels included in one block are not particularlylimited. For example, one block has a square shape including the samenumber of input pixels in vertical and horizontal directions, and thenumber of segmented blocks in vertical and horizontal directions isdetermined to be about eight to 64. The number of segmented blocks maybe fixed or variable depending on conditions for the input image.

Next, in step S702, the image processing unit 218 calculates arepresentative pixel value in each of segmented blocks. Therepresentative pixel value is determined using relatively bright pixelswhich are selected from among a plurality of input pixels included ineach block. One of the simplest methods for calculating therepresentative pixel value is a method of determining a brightest pixelvalue in each block to be the representative pixel value. In anothermethod, a default number of bright pixels are selected in order from thebrightest pixel value in each block, and an average value of theselected pixel values is calculated and used as the representative pixelvalue. In still another method, only pixels having a brightness within adefault range are selected from the brightest pixel value in each block,and an average value of the selected pixel values is calculated and usedas the representative pixel value. Further, the representative pixelvalue may be calculated using a pixel value histogram in each block. Itis also possible to apply a method in which a cluster representativevalue is calculated by clustering pixel values in each block and thebrightest cluster representative value is used as the representativepixel value. The representative pixel value calculated as describedabove is substantially equal to the value obtained by adding the shadowcomponent formed during image capturing to the background level itselfin a case where the background area of the subject is included in theblock. In a case where the background area is not included in the block,the representative pixel value is equal to the value obtained by addingthe shadow component to a relatively bright object area in the block.

As a result of obtaining the representative pixel value for all blocks,in step S703, the image processing unit 218 generates a reduced imagecomposed only of the representative value in each block. The imagecomposed only of the representative pixel value in each block isobtained by reducing the input image to an image having a number ofpixels defined by the number of segmented blocks described above, andthe reduced image is a starting image from which the subsequent shadowestimation processing is started. FIG. 11B illustrates the imagegenerated by reduced image generation processing of step S703 from theinput image illustrated in FIG. 11A.

Next, in step S704, the image processing unit 218 removes elements ofobjects, such as text and pictures, other than the background andshadow, from the reduced image as the starting image, and carries outcorrection processing for leaving only the shadow component. In thiscorrection processing, the representative pixel value in each blockwhich does not include the background area described above is correctedusing the representative pixel value in the adjacent blocks includingthe background area. As a result, only the value obtained by adding theshadow component to the background level is left in the reduced image.The value of a pixel that is abnormally darker than the entire reducedimage or pixel values on the periphery of the pixel is determined to bethe representative pixel value in the block with no background area. Inthis determination, the use of original input image information enablesan improvement in the determination accuracy, using color information inaddition to the brightness of each pixel value. FIG. 11C illustrates animage obtained by removing object elements from the image illustrated inFIG. 11B by the processing of step S704.

In step S705, the image processing unit 218 determines whether the inputimage includes a reflected area, such as reflection of illumination,based on the reduced image (FIG. 11C) from which object elements areremoved. Various determination methods can be applied. For example, ifthe number of pixels having a pixel value equal to or greater than adefault luminance value with respect to the total number of pixels isequal to or more than a predetermined ratio, it is determined that theinput image includes a reflected area. The pixel value having thedefault luminance value or greater is defined by an abnormally brightpixel value that cannot be acquired in normal image capturing.Specifically, the pixel value is a value of a pixel constituting ablown-out highlight area generated when illumination light or the likeis directly or specularly reflected during image capturing. For example,in the case of 8-bit image data, a luminance value of about 250 issuitable. This case is premised on the image captured by the mobileterminal 200 using the camera function including the image capturingunit 203. However, in practice, an image which does not include ashadow, illumination nonuniformity, or the like may be input. In thiscase, the most part of the image is occupied by default values definedby values substantially equal to the luminance value 250 as thebackground level. Accordingly, it is determined that there is areflection, although the image includes no reflected area. To avoidthis, it is preferable to set, for example, an upper-limit value for thepredetermined ratio of the number of pixels, or an upper-limit value forthe entire reduced image, such as an average luminance value, and toprovide a unit for changing the determination result to indicate thatthere is no reflection when such an upper-limit value is exceeded.

In step S706, the image processing unit 218 branches the processingaccording to the result of the determination as to whether there is areflection. If it is determined that there is a reflected area,reflection countermeasure processing of step S707 is executed and thenthe processing proceeds to step S708. If it is determined that there isno reflected area, the processing proceeds to step S703. The reflectioncountermeasure processing executed in step S707 is processing forcorrecting a bright area (outer peripheral area) generated at the outerperiphery of the reflected area in the reduced image generated by theprocessing from step S701 to step S704 illustrated in FIG. 7. The outerperipheral area described herein refers to an area in the vicinity of aboundary between a reflected area and a non-reflected area.

FIG. 8 is a flowchart illustrating the reflection countermeasureprocessing of step S707, and illustrates processing to be sequentiallyperformed on each pixel of the reduced image as the pixel of interest.In practice, the processing is performed by securing an output storagearea for the processing in one of the storage unit 213 and the RAM 211.The description of this mechanism is not provided herein.

In step S801, the image processing unit 218 determines whether the pixelvalue of the pixel of interest is equal to or greater than apredetermined luminance value, and then the processing is branched. Ifthe pixel value is smaller than the predetermined luminance value, thepixel of interest is shifted to the next pixel. If the pixel valueindicates a brightness equal to or higher than the predeterminedluminance value, the processing proceeds to step S802.

In step S802, the image processing unit 218 converts the pixel value ofthe pixel of interest into a lower luminance value. This processingmethod is not particularly limited. For example, processing in which,for example, a pixel value of a pixel of interest is replaced by adarkest pixel value in a range of N×N pixels (e.g., 5×5 pixels) centeredon the pixel of interest is applied. Further, a pixel value of a pixelwith a brightness equal to or higher than a default value may bereplaced by an average luminance value of the reduced image. By theprocessing of step S802, the outer peripheral area of the reflected areais specified and pixel values in the outer peripheral area are convertedinto dark pixel values.

In step S803, the image processing unit 218 determines whether thereflection countermeasure processing is executed on all pixels of thereduced image, and repeats the processing of steps S801 and S802 untilthe processing on all pixels is completed.

FIG. 11D illustrates an image obtained by correcting the imageillustrated in FIG. 11C by the reflection countermeasure processing ofstep S707 described above.

Next, referring again to FIG. 7, in step S708, the image processing unit218 performs enlargement processing on the reduced image by, forexample, a linear interpolation method, so that the number of pixelsbecomes equal to that of the input image. The enlarged image obtained asa result of the processing is an image having the same size as the inputimage obtained by adding the shadow component to the background level,and is a threshold map indicating a distribution of separation levelsbetween the background and the foreground that vary within the image.FIG. 11E illustrates an image obtained as a result of enlarging theimage illustrated in FIG. 11C when it is determined that there is noreflected area in step S706, i.e., a threshold map. FIG. 11F illustratesan image obtained as a result of enlarging the image illustrated in FIG.11D when it is determined that there is a reflected area in step S706.

In this manner, the threshold map is generated from the input image.

FIG. 11G illustrates a result of performing binarization processing onFIG. 11A using FIG. 11E as the threshold map. This is an output imagewhen the reflection countermeasure processing of step S707 is notexecuted, and an unwanted black area appears in a peripheral portion ofthe reflected area due to reflection of illumination or the like. FIG.11H illustrates a result of performing binarization processing on FIG.11A using FIG. 11F as the threshold map. This is an output image whenthe reflection countermeasure processing of step S707 is executed, andthus is an appropriate result.

Reasons for differences in the output results will be described withreference to conceptual diagrams of FIGS. 12A to 12H.

FIG. 12A schematically illustrates an input image, assuming that areflected area is present at a right end where three black lines aredepicted. A thick solid line e-1 in FIG. 12E represents the brightnessof this image on a graph. Three lines of the input image are representedby three thick solid lines, respectively, which extend downward (darkside) from the thick solid line e-1 on the graph. FIG. 12B illustratesan ideal threshold map of FIG. 12A and matches the thick solid line e-1on the graph. When binarization is performed on FIG. 12A using FIG. 12Bas the threshold map, as illustrated in FIG. 12F, an image in whichthree black lines are depicted on a white background is obtained.

FIG. 12C illustrates a threshold map calculated when the reflectioncountermeasure processing is not executed. During calculation of thethreshold map, the reduced image is generated using bright pixel valuesin each of segmented blocks. Accordingly, if there is a bright reflectedarea, the bright area has a property of expanding toward the outside ofthe bright reflected area. Therefore, a bright area which expands fromthe right end in FIG. 12C and is wider than the ideal threshold map ofFIG. 12B is calculated. A broken line e-0 in FIG. 12E represents thebright area on the graph. The bright area represented by the broken linee-0, which is a threshold, is converted into black by binarization, ascompared with the thick solid line e-1 on the graph of the input image.FIG. 12G illustrates a result of binarization in this case. Thiscorresponds to an unwanted black area appearing in FIG. 11G.

FIG. 12D illustrates a threshold map calculated when reflectioncountermeasure processing is executed on FIG. 12C. A broken line e-2 inFIG. 12E represents the threshold map on the graph. Only the bright arearepresented by the broken line e-2, which is a threshold, is convertedinto black by binarization, as compared with the thick solid line e-1 onthe graph of the input image. Accordingly, the binarization resultobtained in this case is appropriate as illustrated in FIG. 12H.

While the binarization processing according to the present exemplaryembodiment is mainly carried out by the image processing unit 218mounted on the mobile terminal 200, processing similar to thebinarization processing can also be executed by the image processingunit 316 or the CPU 311 on the CPU 210 or the MIT 300. Further, theprocessing can also be executed by the computer 104, a server (notillustrated) on a network, and the like.

The execution of the processing as described above enables binarizationprocessing with a high image quality by appropriately changing thethreshold for binarization within one image even when a shadowcomponent, such as a shadow or nonuniformity, is present in an inputimage and a bright area is generated in the image due to reflection ofillumination or the like.

Second Exemplary Embodiment

A second exemplary embodiment illustrates an operation screen flow foreffectively providing threshold map generation processing forbinarization described in the first exemplary embodiment. Descriptionsof contents of the second exemplary embodiment that are common to thecontents of the first exemplary embodiment are not provided herein.

FIG. 10A illustrates an operation screen 110 suitable for a case wherethreshold map generation processing is performed according to the resultof determining, within the apparatus, whether there is a reflection asdescribed in the first exemplary embodiment. An input image to besubjected to binarization processing is displayed in a processing targetimage display area 111 on the operation screen 110. When the user checksthis display and instructs binarization processing, the user presses a“NEXT” button 112. When the “NEXT” button 112 is pressed, if the inputimage includes a reflected area, a threshold map is generated byautomatically executing the countermeasure processing, so that a binaryimage is generated, and then the processing is terminated. In addition,a “CANCEL” button 114 and a “BACK” button 113 are arranged on thescreen, detailed descriptions of which are not provided herein.

FIG. 10B illustrates an operation screen 120 which is configured toenable the user to select valid/valid of reflection countermeasureprocessing, instead of determining whether there is a reflection in thefirst exemplary embodiment. An operation and a processing flow for thisscreen will be described with reference to a flowchart of FIG. 9A.

The user observes the input image to be subjected to binarizationprocessing displayed in a processing target image display area 121, Ifthe image includes no reflection of illumination or the like, the userpresses a “normal mode” button 122. If the image includes a reflectionof illumination or the like, the user presses a “reflection mode” button123. Information about the pressed button is transmitted from theoperation unit 217. The CPU 210 receives the information and transmitsthe information as information indicating valid/in valid of reflectioncountermeasure processing to the image processing unit 218. The imageprocessing unit 218 determines that the “normal mode” button 122 ispressed, the reflection countermeasure processing is invalid, in stepS901, the processing proceeds to step S903 to generate a threshold mapwithout reflection countermeasures. The flowchart of FIG. 7 correspondsto processing in which step S705 is not included and the determinationresult in step S706 indicates “No” without fail. If the “normal mode”button 122 is not pressed in step S901, the processing proceeds to stepS902 and the image processing unit 218 determines whether the“reflection mode” button 123 is pressed. If it is determined that the“reflection mode” button 123 is pressed, i.e., the reflectioncountermeasure processing is valid, the processing proceeds to step S904to generate a threshold map with reflection countermeasures. Theprocessing to be performed when the “reflection mode” button 123 ispressed corresponds to processing in which step S705 illustrated in FIG.7 is not included and the determination result in step S706 indicates“Yes” without fail. In step S905, the image processing unit 218 performsbinarization processing on the input image using the generated thresholdmap.

FIGS. 10C and 10D illustrate operation screens 130 and 140,respectively, which are configured to enable the user to selectvalid/invalid of reflection countermeasure processing while checking abinarization result image. An operation and a processing flow relatingto these screens will be described with reference to a flowchart of FIG.9B.

In the case of carrying out the operation flow, the image processingunit 218 generates threshold maps, which respectively represent thepresence and absence of reflection countermeasures, in advance,generates binary images processed using the threshold maps,respectively, and temporarily stores the two types of binary images inthe storage unit 213. For convenience of description, assume herein thatan image subjected to binarization processing using the threshold mapwith no reflection countermeasures is referred to as a binary image Aand an image subjected to binarization processing using the thresholdmap with reflection countermeasures is referred to as a binary image B.

In steps S906 and S907, like in steps S901 and S902 described above, theCPU 210 waits until the “normal mode” button 122 and the “reflectionmode” button 123 on the screen 120 are pressed and information istransmitted from the operation unit 217. When the “normal mode” button122 is pressed, the processing proceeds to step S908 from step S906 andthe CPU 210 sends an instruction to display the binary image A on thedisplay unit 216. FIG. 10C illustrates the operation screen to bedisplayed in this case. The binary image A is displayed in an imagedisplay area 131 and the “normal mode” is selected. When the “reflectionmode” button 123 is pressed, the processing proceeds to step S909 fromstep S907 and the CPU 210 sends an instruction to display the binaryimage B on the display unit 216. FIG. 10D illustrates the operationscreen to be displayed in this case. The binary image B is displayed inan image display area 141 and the “reflection mode” is selected.

In step S910, the CPU 210 waits until a button on the screen 130 or thescreen 140 is pressed. If an “EXECUTE” button 136 or 146 is pressed, theprocessing proceeds to step S911. The CPU 210 selects the binary imagecorresponding to the previously selected mode and transmits theinformation to the image processing unit 218. Thus, a series ofprocesses are terminated. For example, when the “EXECUTE” button 136 ispressed in a state where the screen 130 is displayed, the binary image Ais selected. When an “EXECUTE” button 146 is pressed in a state wherethe screen 140 is displayed, the binary image B is selected. If the“normal mode” button 132 or 142 is pressed in step S910, the processingproceeds to step S912 and the CPU 210 sends an instruction to displaythe binary image A, i.e., the screen 130, on the display unit 216. Ifthe “reflection mode” button 133 or 143 is pressed, the processingproceeds to step S913 and the CPU 210 sends an instruction to displaythe binary image B, i.e., the screen 140, on the display unit 216.

The provision of the operation unit as described above enables switchingof processing based on the user's determination, for example, as towhether a target image includes a reflected area due to illuminationlight or outside light.

According to the present disclosure, it is possible to performbinarization processing with a high image quality while suppressing theeffect of a shadow component even when the shadow component andillumination light and the like are reflected in a multi-valued imageobtained by capturing a document image.

Other Embodiments

Embodiment(s) of the present disclosure can also be realized by acomputer of a system or apparatus that reads out and executes computerexecutable instructions (e.g., one or more programs) recorded on astorage medium (which may also be referred to more fully as a‘non-transitory computer-readable storage medium’) to perform thefunctions of one or more of the above-described embodiment(s) and/orthat includes one or more circuits (e.g., application specificintegrated circuit (ASIC)) for performing the functions of one or moreof the above-described embodiment(s), and by a method performed by thecomputer of the system or apparatus by, for example, reading out andexecuting the computer executable instructions from the storage mediumto perform the functions of one or more of the above-describedembodiments) and/or controlling the one or more circuits to perform thefunctions of one or more of the above-described embodiment(s), Thecomputer may comprise one or more processors (e.g., central processingunit (CPU), micro processing unit (MPU)) and may include a network ofseparate computers or separate processors to read out and execute thecomputer executable instructions. The computer executable instructionsmay be provided to the computer, for example, from a network or thestorage medium, The storage medium may include, for example, one or moreof a hard disk, a random-access memory (RAM), a read only memory (ROM),a storage of distributed computing systems, an optical disk (such as acompact disc (CD), digital versatile disc (DVD), or Blu-ray Disc (BD)™),a flash memory device, a memory card, and the like.

While the present disclosure has been described with reference toexemplary embodiments, the scope of the following claims are to beaccorded the broadest interpretation so as to encompass all suchmodifications and equivalent structures and functions.

This application claims the benefit of Japanese Patent Application No.2017-171178, filed Sep. 6, 2017, which is hereby incorporated byreference herein in its entirety.

What is claimed is:
 1. An information processing apparatus comprising:an acquisition unit configured to acquire an image representing a shadowcomponent in an image capturing environment, the shadow component beingreflected in a multi-valued image obtained by capturing an image of asubject; a specifying unit configured to specify an area having aluminance greater than a predetermined luminance value, the area beingincluded in the image representing the shadow component acquired by theacquisition unit; a correction unit configured to correct the image insuch a manner that a luminance of an outer peripheral area of the areaspecified by the specifying unit decreases; and a generation unitconfigured to generate a binary image by performing binarizationprocessing on a pixel value of a pixel of interest in the multi-valuedimage based on a pixel value at the same coordinates as those of thepixel of interest in the image corrected by the correction unit.
 2. Theinformation processing apparatus according to claim 1, wherein thecorrection unit corrects the image by replacing a pixel value of thepixel of interest included in the outer peripheral area with a pixelvalue with a lowest luminance value in an area of N×N pixels centered onthe pixel of interest.
 3. The information processing apparatus accordingto claim 1, wherein the correction unit corrects the luminance of theouter peripheral area to reach an average luminance value of the imagerepresenting the shadow component.
 4. The information processingapparatus according to claim 1, wherein, in a case where a pixel valuerepresenting a luminance component of the pixel of interest in themulti-valued image is equal to or greater than a pixel valuerepresenting a luminance component of the pixel of interest in the imagecorrected by the correction unit, the generation unit generates a pixelvalue representing white from the pixel of interest in the multi-valuedimage.
 5. The information processing apparatus according to claim 1,wherein, in a case where a pixel value representing a luminancecomponent of the pixel of interest in the multi-valued image is smallerthan a pixel value representing a luminance component of the pixel ofinterest in the image corrected by the correction unit, the generationunit generates a pixel value representing black from the pixel ofinterest in the multi-valued image.
 6. The information processingapparatus according to claim 1, wherein the pixel value of themulti-valued image includes the shadow component in the image capturingenvironment.
 7. The information processing apparatus according to claim1, wherein the multi-valued image is a gray scale image.
 8. Aninformation processing apparatus comprising: an acquisition unitconfigured to acquire an image representing a shadow component in animage capturing environment, the shadow component being reflected in amulti-valued image obtained by capturing an image of a subject; aspecifying unit configured to specify an area having a luminance greaterthan a predetermined luminance value, the area being included in theimage representing the shadow component acquired by the acquisitionunit; a correction unit configured to correct the image in such a mannerthat a luminance of an area located near a boundary between the areaspecified by the specifying unit and an area not specified by thespecifying unit decreases; and a generation unit configured to generatea binary image by performing binarization processing on a pixel value ofa pixel of interest in the multi-valued image based on a pixel value atthe same coordinates as those of the pixel of interest in the imagecorrected by the correction unit.
 9. An information processing methodcomprising: acquiring an image representing a shadow component in animage capturing environment, the shadow component being reflected in amulti-valued image obtained by capturing an image of a subject;specifying an area having a luminance greater than a predeterminedluminance value, the area being included in the image representing theshadow component acquired in the acquiring; correcting the image in sucha manner that a luminance of an outer peripheral area of the areaspecified in the specifying decreases; and generating a binary image byperforming binarization processing on a pixel value of a pixel ofinterest in the multi-valued image based on a pixel value at the samecoordinates as those of the pixel of interest in the image corrected inthe correcting.
 10. An information processing method comprising:acquiring an image representing a shadow component in an image capturingenvironment, the shadow component being reflected in a multi-valuedimage obtained by capturing an image of a subject; specifying an areahaving a luminance greater than a predetermined luminance value, thearea being included in the image representing the shadow componentacquired in the acquiring; correcting the image in such a manner that aluminance of an area located near a boundary between the area specifiedin the specifying and an area not specified in the specifying decreases;and generating a binary image by performing binarization processing on apixel value of a pixel of interest in the multi-valued image based on apixel value at the same coordinates as those of the pixel of interest inthe image corrected in the correcting.
 11. A non-transitory computerreadable storage medium storing a program executable by a computer toperform an image processing method comprising the steps of: acquiring animage representing a shadow component in an image capturing environment,the shadow component being reflected in a multi-valued image obtained bycapturing an image of a subject; specifying an area having a luminancegreater than a predetermined luminance value, the area being included inthe image representing the shadow component acquired in the acquiring;correcting the image in such a manner that a luminance of an outerperipheral area of the specified area deceases; and generating a binaryimage by performing binarization processing on a pixel value of a pixelof interest in the multi-valued image based on a pixel value at the samecoordinates as those of the pixel of interest in the corrected image.